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ABSTRACT 

The technology of criteria-oriented testing enhanced by the reflexive components is suggested in this article. Tests 
made according to this technology are called academic activity tests. The student chooses or formulates not the answer 
to the problem but an action that is productive in his opinion. So, this type of tests helps not only check the accuracy 
of the achieved results, but traces the course of its getting. The main characteristics of such tests that make it possible 
to estimate it diagnostic potential are presented. Among these are variability of the test and the length of the minimum 
and maximum trajectory of the result achievement. The test of this kind with the discussion of its diagnostic 
possibilities is given as an example 
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INTRODUCTION 


1 “ ' he use of test forms in the educational process is regarded as one of the major issues of pedagogical 
jy / science and practice of the XXI century (Frederickson, Mislevy, & Bejar, 1993, p. 404). The most 
important positive characteristics of test technologies are their objectivity, standardizing of the 
procedures, and better measurability of the outcomes due to the more flexible scale usage. 


Up to the 60-ies of the XX century the test methods used had two target areas: 


• IQ tests; 

• academic achievements tests. 


Both cases are actually referring to the ranking of the testees in accordance with the statistically established norms. 
Hence this approach is called norm-referenced Keeves, 1988). 

As for IQ-tests it should be noted that their contents did not bear much relation to the curriculum, so the correlation 
between the test success and academic achievements was, as a rule, although positive, rather low (less than 0.5). 

Academic achievements tests are a form of knowledge and skills assessment acquired by students in the learning 
process. One of the major disadvantages of this test form is the inability to estimate how the answers were obtained 
by a testee. According to A. N. Mayorov (2000) "... the analysis of the problem solution methods and thinking 
operations that a pupil uses for solving the problems in most cases is hindered or impossible" (p. 38). This proves a 
sheer impossibility of such tests’ usage in assessment of intellectual skills acquired at the time of learning. 

In 1963, Robert Glaser suggested a criterion-referenced assessment as an alternative. This approach is characterized 
by orientation not on the statistical norm but on a given objective substantive standard - standard execution - with 
which the results of each test are compared. The consequence is that in norm-referenced tests a task is not considered 
valid if it is fulfilled by all the testees; in criterion-referenced tests, such a task, it is not only appropriate, but also 
serves as a basis for concluding about the sufficient level of learning material retention by all the students. 

At the initial stage the criterion-referenced tests were aimed at solving the same estimation problem of students’ 
academic achievements. But in the mid-80s as a result of this technology study, it became clear that this approach can 
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be effectively used for the diagnosis of students’ operational and logical activity in the course of tests’ carrying out. 
In other words, it is a diagnosis of students’ mental activity (Starichenko, Egorov, & Yavich, 2013, p. 23). 

CRITERION-REFERENCED TEST 

Here is an example of criterion-referenced test made up for the diagnosis of such a mental action as a definition of the 
classification principle. 

Four tasks are given: three are related to the problems of one kind, and one is related to another. They are denoted as 
A, B, C, D. 

A. A Farmer planes on plowing 150 acres daily. However, due to the good weather, he was able to plow up 
to 25% daily, and therefore, all the work was completed one day earlier than expected. How long did it 
take him to plough the field? 

B. The computer-controlled machine manufactures 25 details per hour. After optimizing of the program, it 
started to manufacture 3 more details per hour. It made it possible to fulfill an order three hours faster. 
What was the order volume? 

C. It takes the train 1.2 hours to pass the distance between the two stations. Because of track repair the train 
decreased its speed by 20% and went the distance in 1.5 hours. Find the initial speed of the train. 

D. Two workers were set to make 8,840 details. Because of the difference in abilities the productivity of 
the first worker is 35% higher than of the second one. After four days’ work, the first worker was 
transferred to another job. It took the second worker one more day to fulfill the order. How many details 
were produced by each worker? 

The essential feature of this test is that while performing it, a student does not solve the problems but compares and 
combines the tasks on the basis of generality of the relation types between values (task D is an odd one out). The 
generality of the plot (do some work - tasks A, B, D), the similarity of the individual parts (the relationship between 
the values of the quantities given in the form of percentages - tasks A, B, D) are not sufficient grounds for concluding 
that the tasks are related to one another and are the same type. Students’ choice of classification criterion makes it 
possible to estimate their analytical mental activity. 

However, the solution of the problem doesn’t reduce to the classification procedure, it unfolds as a holistic sequence 
of mental operations. In our opinion, for the effective diagnosis of such activities, the technology of academic activity 
tests, which has been developed since the early 90s of the last century should be used. The extensive study of this 
technology was held by Senognoeva (2013). However, this technology should be completed with some tools of 
reflexive nature for the diagnosis of the academic activity, not just its results. 

The gist of the academic activity test technology is that unlike the standard test, where the student should choose one 
of the answers or just write down the result he got (after that the teacher can only form a hypothesis why the student 
got such a result), the academic activity test suggests choosing one operation of several alternatives at every step. 
Every time an operation is estimated to be a productive one (i.e. leading to the problem solution) or a dead-end. As a 
result of these steps a solution is arrived at, i.e. the student always has a solved problem as a result. This is very 
important from the psychological point of view. 

The principal characteristic of an academic activity test is that there are always different ways of solution approaches 
to one and the same problem. These ways of solution may be of different length depending on the choice of the theory 
that the student will use for the problem solution, on the level of his knowledge and skills etc. so the academic activity 
tests could be regarded as one of the paradigms of the adaptive training systems (Senognoeva, 2013); however, the 
adaptation is not effected by the level of difficulty, but estimates whether the mental action is correct. 

It is important that the choice of action at one or another stage of the test is the projection of an appropriate mental 
action. We can note how well the student can select the information and separate essential from the nonessential, 
convert it to sign-symbolic form and operate with it, his mental actions being expanded or contracted. 
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The fundamental development of the academic tests technology in our work is the inclusion of items allowing the 
students to carry out reflection of their actions. The fact is that the competence formation cannot be fully realized if 
the student doesn’t self-reflect his activity as universally valid, i.e. as applicable to the different starting situations. 

Note that this «reflexive» addition to the test tasks brings them to better correspondence with the psychological- 
pedagogical concept of “academic activity”. By Davydov’s (1982) interpretation, an academic activity is the one of a 
subject in mastering of the methodology of learning actions and self-development in the process of training problems 
solution on the base of external monitoring and estimation, turning into self-control and self-estimation (Keller, 
Yavich, Domoshnitsky, 2014, p. 133). 

In the light of the foregoing we will mention that the solution of training problems is not a goal but a means of the 
goals’ achievement, so the role of a training problem is that the students master the common ways of working with 
the material and transfer them to the solution of the problems where these methods are applicable. In the problem that 
represents a complex system of information about some phenomenon, object, process, only some facts are given. The 
rest information is unknown and should be found out on the basis of the other data, through the establishment of the 
links between them, transformations fulfillment etc. 

So, the method of a problem solution is any procedure that leads to the task solution. In other words, the way of a 
problem solution correlates the subjective characteristics of the person and problem solver that determine both the 
selection of the operations sequence and the general strategy of the solution. The solution of the problem in different 
ways provides more opportunities for the development of a student. While solving the problem by the only one method 
the goal of the student is to find the correct answer; using several ways of the problem solution he faces the choice of 
the shortest and most efficient one. It requires the actualization of theoretical knowledge and many known techniques 
and methods or the creation of the new methods and techniques applicable to the situation in the problem. Herewith 
the students accumulate experience of knowledge application that contributes to the development of logical search 
techniques and, in turn, develops their research abilities. 

In the process of looking for solution, a student can or often has to try different strategies and methods. The algorithm 
of problem solution is a chain of interrelated, serendipitously chosen statements. Such chain building is one of the 
main academic skills, the basis of the scientific activity. The final step of the solution is the choice of the strategy, 
realized by the definite way of proceeding. The pattern of a training problem solution together with the tentative one 
includes other parts of the mode of action, primarily controlling and executive. Unimpaired operation of an academic 
activity assumes well-formedness of all the parts of the mode of action. 

We can single out the following parts of the mode of action: 

• Finding the backgrounds for the academic activity success (the student should realize what he knows 
and is able to do and the way it correlates with the problem) 

• The choice of the general strategy 

• Determination of the missing information 

• Search for the missing information 

• Search for confirmatory evidence for the chosen way of solution propriety 

• Decomposition of the main problem into the sub-problems 

• Solution of the sub-problems 

• Choice of the most effective algorithm of actions 

• Execution of the operations appropriate in this particular case. 

Therefore, the problem solution goes through the realization of some academic actions sequence. The student should 
have an idea about the academic activity components, the ways of these actions carrying out and their sequence. 

Knowledgeable fulfillment of an academic task at all stages and timely correction of the errors are achieved by means 
of step-by-step checking. A student must not only identify the difficulty that arose in the course of solving the problem, 
but also realize to what step of solution it refers. For academic activities tests it is very important that the step-by-step 
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checking can refer both to the carried out and to planned actions. In turn, it makes it possible to analyze the work on 
the task before its actual execution, to anticipate the results. The students, who mastered such form of control, do not 
start the task execution the moment they get it. At first they make an action plan, compare and estimate the ways of 
solution, choose the most appropriate ones, think over the possible consequences of their usage and only after that set 
out on the task. If a need arises, they can return to every step of the task fulfillment, check the carried out actions, find 
the false ones and correct them. Academic activity becomes autonomous and self-regulatory. A student, having 
mastered such activities is able to foresee the occurrence of possible errors and difficulties and to schedule timely 
measures to prevent them. 

As previously stated, the task in the academic activity test is a set of different ways, each leading to the solution of the 
problem, but these ways are different in composition of the actions, suggested for their fulfillment. 

The first step along the way of the test task is the elicitation of all the concepts used in the stated problem. The direction 
of the next step is chosen by the student depending on how deeply he/she has understood the contents of each of the 
selected concepts and how he understands the interrelations of the concepts. These links may already be known or can 
be easily established by the student due to their knowledge and skills or may turn out to be an auxiliary task that will 
require the straight-line diagram of this relation. Similarly, the second step is carried out, etc. Therefore, each way has 
its own length. The shortest way corresponds to be degree of student’s being familiar with the material at the highest 
level of convolution. If on a particular step a subtask occurs that requires a separate decision, it is carried out after the 
same pattern. In other words, the problems in the tests have the structure of a fractal type. 

The ways of the problem solution, the level of its convolution to a great extent depend not only on the problem itself, 
but on the facts and methods learnt by the student by the time of the test. 

An additional advantage of testing is possible using interactive computer technologies. The implementation of 
academic activity tests without computers is next to impossible. Firstly, the goal of the test fulfillment is not the 
obtainment by the student of a single answer, but finding a rather long way where the subsequent step depends on the 
previous one and the analysis of every selected answer should be presented to the student so that he did not know the 
analysis of the other answers. Secondly, for a full analysis it is necessary to have a complete record of the test task 
fulfillment for each student. Third, due to the considerable volume of the fixed information about the test fulfillment 
it is necessary to have the set of tools information processing implemented in the form of computer software. 

We will cite the example of the training test. For easy perception we will consider a fairly simple task of school 
geometry (in particular, it means that students do not know the trigonometric functions of the angle and cannot use 
them to solve this problem). 

The students should solve a text problem. The testing is computer-based; the presentation of the test tasks, the variants 
of answers and the analysis of the students’ choice are implemented to interactively. To describe the order of the 
assignments presentation with all the alternatives and analysis we use the following techniques: the tasks are presented 
to the students consistently (they are numbered from 1 to 24 - this is the total number of tasks in this test), the answers 
to every task are marked by the letters of the alphabet. After choosing the answer the student is given an analysis and 
the instructions on what to do next (the analysis is marked by a combination of the letter D and the analysis serial 
number). In addition, we provide diagnostic comments about the nature and the level of learning actions development. 
Students do not see those comments, but the teacher may obtain them; access to them is protected by a teacher’s 
individual password. 


THE PROBLEM 

The diagonal AC of the parallelogram AB CD makes the angle 30° with the side AB. Solve the area of the parallelogram 
if AB = 6 cm and AC = 8 cm. 
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Instruction 

To solve the problem, you should execute the steps in sequence; starting with p. 1 and each time choose one of the 
suggested answers. 


P. 1. For the solution of the problem, the following information will be useful: 


A. The definition of a parallelogram. 

-> 

Dl 

B. The properties of a parallelogram. 


D2 

C. The formula for solving the area of a triangle. 


D3 

D. The formula for solving the area of a parallelogram. 


D4 

E.Other information. 


D5 


Dl. Remember the definition and think whether it can help you to define other parameters of the parallelogram. 
Consider another variant. Return to p. 1 

D2. This way is possible. Go to p. 2. 

D3. This way is possible. Go to p. 11. 

D4. This way is possible. Go to p. 12. 

D5. Write down your own version in the notebook and discuss with the teacher wants to do next. 

Diagnostic Comment 

If a student chooses A then it is indicated that in this task they will be guided on only outward sign: on the fact that 
the parallelogram is mentioned in the problem. Here we can diagnose insufficient formation of such a universal mental 
action as the ability to identify and formulate essential information for this problem solution. 

If the student chooses B then it is likely that the overall plan of solution is still absent, but they understand that the 
definition of a parallelogram is unlikely to be helpful in itself, and assumes that there are properties that allow to them 
to solve the problem. 

If the student chooses C then, although such choice may lead to the problem solution, it is likely that this choice is 
made because the triangle ABC is defined clearly, i.e. all its characteristics, including the area, can be calculated one 
way or another. It is pretty typical but in general it is a counterproductive strategy of trials and errors, when instead of 
trying to make a general plan of the goal achievement, a student takes steps at random simply because they are feasible. 

The choice of D is strategically correct since as it is known making a plan for a problem solution should be started 
with the analysis of what you want to solve (Chelyshkova, 2001, p. 165). In this case, the area of the parallelogram is 
being solved so the choice of the answer is quite natural. However, this does not mean that the student already has a 
full plan for the problem solution, the final conclusion can be made after fulfillment of the subsequent test tasks. 

The choice of E will not lead to the solution. In this case the diagnosis is the same as that shown in Dl, of course, with 
the adapted commentary why that choice is untoward. 


P. 2. What property of the parallelogram is useful for the problem solution in your opinion? 


A. The opposite sides of a parallelogram are equal. 

-> 

D6 

B. A diagonal of a parallelogram divides it into two equal triangles. 

-> 

D7 

C. The diagonals of a parallelogram bisect at the cross point. 

-> 

D6 

D. Other property. 

-> 

D8 
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D6. Think whether this property will help to define some other parameters of the parallelogram. Consider another 
variant. Return to p. 2. 

D7. This is useful information. Go to p. 3. 

D8. Write down your own version in the notebook and discuss with the teacher what to do next. 

Diagnostic Comment 

If the student chooses the B, this enhances the likelihood of a hypothesis that he already has a plan of the problem 
solution. If he chooses A or C then it means that he has no plan. If he chooses D, then either he gives a formulation 
which is equivalent to B but he does not understand it (this shows the problems of his logical thinking), or his 
formulation is equivalent to the choice of A or C with the corresponding diagnostics. 


P. 3. To solve the area of a triangle it is enough to know: 


A. The lengths of all sides of the triangle. 

-> 

D9 

B. The length of the side and the height dropped on this side. 

-> 

DIO 

C. The lengths of the two sides of the triangle and the angle between them. 

-> 

D9 

D. Other numerical information on the triangle. 

-> 

Dll 


D9. It's correct, but your knowledge for needed calculations is not enough. Consider another variant. Return to p. 3. 
DIO. It's correct. Go to p. 4. 

Dll. Write down your own version in the notebook and discuss with the teacher what to do next. 

Diagnostic Comment 

As in p. 2, the choice of the correct alternative enhances the likelihood of a hypothesis that the student already has a 
full plan the problem solution; the choice of the incorrect alternative indicates faulty interpretation of his knowledge 
for the problem solution. 


P. 4. To solve the area of the triangle ABC you choose: 


A. The side AB and the height dropped on it. 

-» 

D12 

B. The side BC and the height dropped on it. 

-» 

D13 

C. The side AC and the height dropped on it. 

-» 

D14 


D12. This way is possible. Go to p. 5. 

D13. This way is possible but your knowledge is not enough for the necessary calculations. Consider another variant. 
Return to p. 4. 

D14. This way is possible. Go to p. 6. 

Diagnostic Comment 

Here is no variant with the appeal to the teacher because the choice of the alternatives is exhausted. The actions 
suggested in A and C lead to the problem solution, since we can say with confidence that student has a developed plan 
for the problem solution. For the variant D13 the comment is the same as for D9. 
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P.5. To solve the height CH dropped from the vertex C on the side AB you will consider: 


A. The right triangle ACH. 


D15 

B. The right triangle BCH. 


D16 

C. Another geometric figure. 


D17 


P. 6. To solve the height BH dropped from the vertex B on the side AC you will consider 


A. The right triangle ABH. 

-> 

D15 

B. The right triangle BCH. 

—> 

D16 

C. Another geometric figure. 

—> 

D17 


D15. It's correct. Go to p. 7. 

D16. Note that in this case you will not be able to use the information on the angular size BAC. Consider another 
variant. Go to the same point. 

D17. Write down your own version in a notebook and discuss with the teacher the choice of next point. 

Diagnostic Comment 

If A or B have been chosen, then purely technical part of the problem solution is remained. The choice of C means, 
that actually all the previous steps have been taken at random. 


P. 7. The cathetus of the right triangle lying opposite to the angle of 30° is equal 

A. The half of the hypotenuse of the triangle. 


D18 

B. Another cathetus. 


D19 

C. The half of the another cathetus. 

-> 

D19 

D. Other value, different from the mentioned in A - C. 

-> 

D20 


D18. It's correct. Go to p. 8. 

D19. You don’t know the property of the cathetus lying opposite to the angle of 30°. Consider another variant. Return 
to p. 7. 

D20. Write down your own version in a notebook and discuss with the teacher the choice of next point. 

Diagnostic Comment 

It is testing of the students’ actual knowledge. If a student chooses D, then either he gives a formulation which is 
equivalent to A, but he does not understand it (this indicates the problems in his logical thinking), or his formulation 
is equivalent to B or C with the corresponding diagnostics. 


P. 8. The area of the triangle is equal to 


A. The product of the triangle height by the base. 

—» 

D21 

B. The half the product of the triangle height by the base. 

— » 

D22 

C. The value calculated differently than described in A or B. 

— » 

D23 


D21. You don’t know the triangle area formula. Consider another variant of answer. Return to p. 8. 
D22. It is correct. Go to p. 9. 

D23. Write down your own version in a notebook and discuss with the teacher the choice of next point. 
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Diagnostic Comment 

Here as in the previous task, the student’s actual knowledge is examined. 

1. Calculate the area of the triangle ABC and enter the answer:_. 

If the answer is correct then transfer to p. 10, otherwise return to p. 9. 

2. Calculate the area of the parallelogram ABCD and enter the answer:_. 

If the answer is correct then the fulfillment of the test task is completed, otherwise return to p. 10. 


P. 9. How is the area of the triangle ABC and area of a parallelogram ABCD interrelated? 


A. The area of the triangle is equal to half the area of a parallelogram. 


D24 

B. The area of the triangle is a quarter of the area of a parallelogram. 


D25 

C. The relation between these values depends on the shape of a parallelogram - whether 
it is a rectangle, rhombus, etc. 

- 

D25 


D24. It's correct. Go to p. 3. 

D25. You do not know the correct interrelation between the areas of the triangle and the parallelogram. Consider 
another variant. Go to p. 11. 

Diagnostic Comment 

The choice of A or B means that the student has an overall plan of the problem solution, but in the case B it would 
lead to an erroneous result. The choice of p. C confirms the hypothesis about the choice of the answer in p. 1 
exclusively oriented on the source data rather than on that is necessary to solve in the problem. 


P. 10. To solve the area of a parallelogram it is sufficient to know 

A. The lengths of all sides of the parallelogram. 

->■ 

D26 

B. The length of one of the sides and the height dropped on this side 

->■ 

D27 

C. The lengths of two adjacent sides of the parallelogram and the angle between them. 

—> 

D28 

D. Other numerical information on the parallelogram. 

—> 

D29 


D26. It isn’t correct; remember that the parallelogram is not a rigid figure. Consider another variant. Go to p. 12. 
D27. It's correct. Go to p. 13. 

D28. It's correct, but your knowledge is not enough for the necessary calculations. Consider another variant of answer. 
Go to p. 12. 

D29. Write down your own version in a notebook and discuss with the teacher the choice of next point. 

Diagnostic Comment 

The choice of A means a random walk in the search of a solution. The choice of the correct alternative B enhances the 
likelihood of a hypothesis that the student already has a full plan of the problem solution. The choice of the alternative, 
C, indicates misestimating of the student’s knowledge necessary for this problem solution. 
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P. 11. To solve the area of the parallelogram you choose: 


A. The side AB and the height dropped on it. 

-» 

D30 

B. The side BC and the height dropped on it. 

-> 

D31 

C. The side CD and the height dropped on it. 

-> 

D32 

D. The side AD and the height dropped on it. 

-> 

D31 


D30. This way is possible. Go to p. 14. 

D31. Note that in this case you will not be able to use the information on the value of the angle BA C. Consider another 
variant. Go to p. 13. 

D32. This way is possible. Go to p. 17. 

Diagnostic Comment 

The choice of B or D means that the student still has no any clear plan for the problem solution, it is likely that he is 
moving on the trajectory that is paved by this training test, without clear awareness of the performed actions goal. If 
a student chooses A or C, then the degree of the planned awareness will be shown in the process of the subsequent 
tasks fulfillment. 


P. 12. To solve the height CH dropped from the vertex C on the side AB you will consider 


A. The right triangle ACH. 

—> 

D33 

B. The right triangle BCH. 

—> 

D16 

C. Another geometric figure. 

-> 

D17 


D33. It's correct. Go to p. 15. 

Diagnostic Comment 

This paragraph coincides with p. 5, so the comments are the same. We have to use the diagnostics D33 (instead of 
D15), since the subsequent trajectory will be different than after p. 5. 


P. 13. The cathetus of a right triangle lying opposite to the angle of 30° is equal to 

A. The half of the hypotenuse of the triangle. 

-> 

D34 

B. Another cathetus. 

-> 

D19 

C. The half of the another cathetus. 

-> 

D19 

D. Other value, different from the mentioned in A - C. 

-> 

D20 


D34. It's correct. Go to p. 10. 

Diagnostic Comment 

This paragraph coincides with p. 7, so the comments are the same. We have to use the diagnostics D34 (instead of 
D18), since the subsequent trajectory will be different than after p. 7. 


P. 14. To solve the length of the side CD it is necessary to use 


A. the properties of the opposite sides of the parallelogram. 

-> 

D35 

B. the properties of the triangles into which a diagonal divides a parallelogram. 

-> 

D35 

C. the properties of triangles into which the diagonals divide the parallelogram. 

-> 

D35 


D35. This way is possible. Go to p. 17. 
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Diagnostic Comment 

In all cases the conclusion of the equality of the sides AB and CD is made, but in fact, this way is the elongation of the 
way 13-14-15-10. 


P. 14. To solve the height AH dropped from the vertex A on the side CD you will consider 


A. The right triangle ACH. 


D36 

B. The right triangle ADH. 


D16 

C. Another geometric figure. 


D17 


D36. It's correct. Go to p. 15. 

Diagnostic Comment 

Essentially this point coincides with p. 14, so the comments are the same. 

RESULTS 

To visualize the set of trajectories of the problem solutions, it is convenient to represent them by a directed graph. The 
vertices are the paragraphs of test task, and the ribs are the transitions from one paragraph to another. For the 
considered test task the corresponding graph is shown in Figure 1. 

Figure 1. The structure of the academic activities test 



One of the important objective characteristics of the academic activity test is the number of possible ways (without 
loops), leading from the initial vertex to the end vertex. It is naturally called the variability of the test (Gorbachova 
(1988) called it the width of the test) since it shows a variety of possible approaches to the problem solution provided 
in the test. Such variety makes it possible to display the individual characteristics of students’ thinking as well as the 
peculiarities of practical usage of their knowledge and skills. Our experiments with academic activity tests show that 
the good test’s range of variability should be no smaller than three. Flowever, if variability is too big, it is undesirable 
and the student’s test results are difficult to compare. In our opinion, the upper limit of variability should not exceed 
eight. The variability of the reviewed test is equal to six, thus, it is within the boundaries required by this parameter. 
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The second objective characteristic has a complex character and is described by three values: the length of the longest 
way from the initial vertex to the end (without loops), the length of the shortest way between the vertices and the ratio 
of these numbers. The meaning of this characteristic is clear: it allows us to estimate the extent to which the student 
is able to choose the best problem solution. It is clear that the ratio should be larger than 1 otherwise the test does not 
diagnose this student’s ability. For our test this ratio is 1.8; in our view this is an acceptable value for the diagnosis. 
But the establishment of the optimal limits for this parameter requires further research. 

With regard to the length of the longest way between the start and end vertices the choice of the optimal range for this 
value depends on of students’ age and the level of their intelligence development. 

The matter already mentioned, the actual length of the test fulfillment depends on how many times the student had to 
go back to the same paragraph until he found a productive continuation. The ratio of actual length to the nominal 
length shows to what extent the student solved the problem consciously, i.e. the level of appropriate mental actions 
development. The maximum possible value of the actual length can be determined a priori for each way from the 
initial vertex to the end vertex. The ratio of this value to the nominal length of the route is natural to call the diagnostic 
ability of test task. For our task this characteristic for various routes averaged 2.3. Our experiments show that good 
separation of students according to these diagnostics occurs if this characteristic lies in the range from 1.8 to 3. 

DISCUSSION AND CONCLUSIONS 

Theoretical underpinning and experimental proof let us make the following conclusions: 

1. The suggested technology (unlike tests that are designed only to get an answer) makes it possible to 
follow the path of the student’s mental actions in the process of the problem solution. It provides an 
opportunity to state the necessary pedagogical techniques to improve the learning efficiency. 

2. For traditional ways of testing such important characteristics as reliability, validity etc., that help to 
differentiate the students according to their academic achievements are set a posteriori, that is based on 
the results of the tests of the control group. The suggested technology allows to estimate these 
characteristics a priori 

3. The suggested technology besides the diagnostic has also a learning effect. While test undergoing the 
student has to analyse his strategy for the problem solution, choosing the most productive way. 

Currently the suggested technology has worked out to greater extent for Mathematics and Science. Its usage for 
Economics, Social science and Humanities requires an additional research. 
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